[BCI] QVAC-17057 feat: add bci-whispercpp POC with batch transcription#1583
Merged
Conversation
3 tasks
3 tasks
Contributor
Tier-based Approval Status |
This was referenced Apr 15, 2026
3 tasks
GustavoA1604
requested changes
Apr 15, 2026
GustavoA1604
left a comment
Contributor
There was a problem hiding this comment.
Need to run JS lint as well
This was referenced Apr 16, 2026
sharmaraju352
pushed a commit
that referenced
this pull request
Apr 20, 2026
…n, fix Linux linkage
- Refactor BCIWhispercpp to use createJobHandler + exclusiveRunQueue
from @qvac/infer-base instead of manual promise plumbing, matching
the TranscriptionWhispercpp / LlmLlamacpp addon pattern
- Constructor now takes { files: { model }, logger, opts } (was { modelPath })
- transcribe/transcribeFile/transcribeStream return QvacResponse
- Add unload(), getState(), exclusiveRunQueue-serialized destroy()
- Add @qvac/infer-base dependency
Address all review feedback from Gustavo (PR #1583):
- Remove unused END_OF_INPUT, totalSamples_, sleep_for(1ms)
- Use QvacErrorAddonBCI for model-not-found, add BUFFER_LIMIT_EXCEEDED
- Fix n_threads/duration_ms double→int conversion in BCIConfig.cpp
- Add bounds validation for all BCIConfig numeric params
- Throw on unknown config keys (was silently ignored)
- Consume gpu_device in context params
- Collect whisper timings in runtimeStats()
- Trim unused BCIErrors enum values, map codes to distinct names
- Add MAX_BUFFERED_BYTES guard and nextSafeId in bci.js
- Fix _activeJobId race: set after native acceptance
- Remove unimplemented bciConfig params from JS whitelist + index.d.ts
- Promote hardcoded kernel-trim threshold to named constant
- Pre-allocate dummyAudioPad_ as class member (avoid repeated allocs)
- Rename bci-addon.test.js → addon.test.js
- Replace t.skip() with proper assertions
- Fix day_idx handling in tests/examples (group by day, pass to config)
- Generate comprehensive NOTICE file
- Update vcpkg overlay to v1.8.4 description
Fix Linux C++ test linkage:
- Add vcpkg triplets (x64-linux, arm64-linux) with -stdlib=libc++
- Add linux-clang toolchain (clang-19)
- Set VCPKG_OVERLAY_TRIPLETS in CMakeLists.txt for Linux builds
Made-with: Cursor
added 3 commits
April 20, 2026 16:25
…nscription Add a new @qvac/bci-whispercpp addon that transcribes brain-computer interface neural signals into text using a modified whisper.cpp backend. This POC includes: - C++ native addon with BCI model inference (NeuralProcessor, BCIModel, BCIConfig) built on the qvac addon-cpp framework - CMake + vcpkg build system with whisper-cpp overlay ports carrying BCI-specific patches (variable conv1 kernel, windowed attention) - JavaScript API: BCIWhispercpp class with batch transcribeFile/transcribe - Integration tests for load/destroy and batch transcription - Example script and model conversion tooling - WER utility for accuracy measurement Streaming transcription will be added in a follow-up PR (QVAC-17062). Made-with: Cursor
…n, fix Linux linkage
- Refactor BCIWhispercpp to use createJobHandler + exclusiveRunQueue
from @qvac/infer-base instead of manual promise plumbing, matching
the TranscriptionWhispercpp / LlmLlamacpp addon pattern
- Constructor now takes { files: { model }, logger, opts } (was { modelPath })
- transcribe/transcribeFile return QvacResponse
- Add unload(), getState(), exclusiveRunQueue-serialized destroy()
- Add @qvac/infer-base dependency
Address all review feedback from Gustavo (PR #1583):
- Remove unused END_OF_INPUT, totalSamples_, sleep_for(1ms)
- Use QvacErrorAddonBCI for model-not-found, add BUFFER_LIMIT_EXCEEDED
- Fix n_threads/duration_ms double→int conversion in BCIConfig.cpp
- Add bounds validation for all BCIConfig numeric params
- Throw on unknown config keys (was silently ignored)
- Consume gpu_device in context params
- Collect whisper timings in runtimeStats()
- Trim unused BCIErrors enum values, map codes to distinct names
- Add MAX_BUFFERED_BYTES guard and nextSafeId in bci.js
- Fix _activeJobId race: set after native acceptance
- Remove unimplemented bciConfig params from JS whitelist + index.d.ts
- Promote hardcoded kernel-trim threshold to named constant
- Pre-allocate dummyAudioPad_ as class member (avoid repeated allocs)
- Rename bci-addon.test.js → addon.test.js
- Replace t.skip() with proper assertions
- Fix day_idx handling in tests/examples (group by day, pass to config)
- Generate comprehensive NOTICE file
- Update vcpkg overlay to v1.8.4 description
Fix Linux C++ test linkage:
- Add vcpkg triplets (x64-linux, arm64-linux) with -stdlib=libc++
- Add linux-clang toolchain (clang-19)
- Set VCPKG_OVERLAY_TRIPLETS in CMakeLists.txt for Linux builds
Made-with: Cursor
…ayer flash attn Update whisper-cpp overlay to 5645ad60 which includes: - Cached window_mask recompute for exp_n_audio_ctx overrides - Per-layer flash attention (upper encoder layers use FA even with BCI) - std::abs instead of C abs in mask computation Made-with: Cursor
d2d08ed to
0eec4b4
Compare
Update overlay to tetherto/qvac-ext-lib-whisper.cpp@3e91e3a4 which addresses jpgaribotti's review on PR #10: 1. Extract compute_window_mask() helper to eliminate duplicated O(n_ctx^2) mask fill logic 2. Guard encode-time mask block with hparams.is_bci 3. Add is_bci to graph builder window_mask guard 4. Validate BCI hparams (conv1_kernel > 0, window_size >= 0) 5. Document n_mels > 256 threshold convention Bump port-version to 3. Made-with: Cursor
Address Gustavo's review feedback: test fixtures (neural_sample_*.bin) are gitignored but the PR had no way for developers to obtain them. Rewrite download-models.sh to fetch both models and test fixtures from the bci-test-assets-v0.1.0 GitHub release. Supports --models, --fixtures, or both (default). Made-with: Cursor
…cleanup - Bump whisper-cpp override in vcpkg.json from 1.7.5.1 to 1.8.4 to match the overlay port version - Move gtest to a vcpkg "tests" feature so it is only pulled when BUILD_TESTING=ON - Fix PaddedFramesAreZero test: use mel-major indexing (data[bin * n_frames + frame]) matching the actual processToMel layout - Remove four unused overlay patch files (0001–0004) now that portfile.cmake fetches from the tetherto fork with patches baked in - Add TODO comment in download-models.sh noting the temporary personal fork for release assets Made-with: Cursor
…docs accuracy
- Wrap transcribe() in exclusiveRunQueue to prevent race between
inference and unload/destroy
- Use find_last_of("/\\") in loadEmbedderIfNeeded for Windows compat
- Add empty-buffer guard in bci.js append() before end-of-job
- Update download-models.sh to use tetherto/qvac release repo
- Add transformers to NOTICE and README model conversion prerequisites
- Fix README WER table to match actual live test results (6.0% avg)
- Fix BCI_V184_COMPAT.md stale test filename and overlay ref
- Remove unused bci_wer_vs_expected field from manifest.json
- Update whisper.cpp patches section to reflect fork-based overlay
Made-with: Cursor
- Fix unload/destroy race: call destroyInstance() before _job.fail() so the native side stops before the JS job is failed, and remove redundant cancel() call (destroyInstance already cancels internally) - Wrap BCIInterface construction in try/catch so a native init failure sets addon=null and throws a structured QvacErrorAddonBCI - Change JSAdapter loadContextParams/loadMiscParams/loadBCIParams to return void (callers already mutate via reference, return was dead) - Add dayIdx bounds-check warning in BCIModel::process when the value falls outside [0, numDays-1] before silent clamping - Promote hardcoded gaussian smoothing params (std=2.0, kernel=100) to named constants K_SMOOTH_KERNEL_STD / K_SMOOTH_KERNEL_SIZE - Add NeuralProcessor::getNumDays() accessor for the bounds check - Remove [key: string]: unknown escape hatch from WhisperConfig in index.d.ts; enumerate all valid keys explicitly - Fix test:cpp:run script to use direct path instead of cd && chain Made-with: Cursor
ogad-tether
reviewed
Apr 20, 2026
ogad-tether
requested changes
Apr 20, 2026
ogad-tether
left a comment
Contributor
There was a problem hiding this comment.
I found two blocking issues and left inline comments with details: 1) transcribe() currently releases the exclusive-run slot before the returned response settles, which can stale an in-flight response and still trip JOB_ALREADY_RUNNING on the native side under concurrent calls; and 2) the published exports map does not match the low-level subpaths documented in the README, so those imports will fail after publish.
Address ogad-tether review feedback on PR #1583: 1. Inference queue: transcribe() now holds its slot until the response settles via _enqueueInference(), matching the pattern from TranscriptionWhispercpp._enqueueExclusiveRunResponse(). Previously the exclusiveRunQueue released the slot as soon as runJob() was accepted, allowing a second concurrent transcribe() to race in and either clobber the first response or get rejected by the native side. 2. Exports map: add ./bci, ./bci.js, and ./binding subpath exports so the low-level BCIInterface API documented in the README is accessible after publish. The exports map previously only exposed ./binding.js, blocking require('@qvac/bci-whispercpp/bci'). Made-with: Cursor
…s review Align the BCI package with the inference-addon family conventions and resolve the review findings that accumulated across PR #1583. Breaking changes - Package directory renamed from packages/bci-whispercpp to packages/qvac-lib-infer-bci-whispercpp (npm name @qvac/bci-whispercpp unchanged). - Error codes moved from 7001-7013 (collided with @qvac/tts-onnx and the @qvac/transcription-parakeet fallback range) to the dedicated 26001-27000 range. Also adds FAILED_TO_START_JOB, INVALID_CONFIG, and EMBEDDER_WEIGHTS_INVALID for cases that were previously swallowed. Pattern / standard alignment with peer addons - Add addonLogging.js + addonLogging.d.ts + ./addonLogging subpath export. - Add CHANGELOG.md, PULL_REQUEST_TEMPLATE.md, tsconfig.dts.json. - Pin qvac-lib-inference-addon-cpp vcpkg dep to 1.1.5#1 (port-version). - vcpkg default-registry switched from git@github.com: to https:// (fixes anonymous clones and CI runners without an SSH deploy key). - Lint glob now covers lib/**/*.js. - bare engine bumped from >=1.19.0 to >=1.24.0 to match llamacpp-llm/embed. - VCPKG_OVERLAY_TRIPLETS set unconditionally and preserves external value. - Remove test:unit script that pointed at a non-existent dir; add build:pack, lint-cpp, test:dts scripts matching peer conventions. - package.json files array now includes README.md, CHANGELOG.md, and addonLogging artifacts; repository.directory + homepage point at the renamed path. PR review fixes (Gustavo, ogad-tether, github-code-quality bot) - day_idx default aligned: C++ runtime default is now 0 (matches the public JS/TS docs and NeuralProcessor header default). - BCIInterface.runJob rewrap now uses FAILED_TO_START_JOB instead of the misleading FAILED_TO_APPEND; input is validated (Uint8Array, non-empty). - day_idx: -1 passthrough mode is now explicitly documented in configChecker, README, and index.d.ts, and values < -1 are rejected at the JS boundary. - JS _load no longer sets suppress_nst/temperature defaults that fought the BCI-tuned C++ defaults in toWhisperFullParams. - Duplicate checkConfig call in BCIWhispercpp._load removed; validation now happens once inside the BCIInterface constructor. - whisper_log_set guarded by std::once_flag so it does not clobber any log handler a coexisting whisper-based addon installed in the same process. - Embedder weight loader now checks the stream state after every read and returns false on truncation instead of silently marking the weights as loaded and producing garbage at inference time. - NeuralProcessor day projection is now memoized per day_idx; same-day batch inference no longer rebuilds the O(nf^2 * r) dense matrix. - cancelRequested_.store(false) now runs before reset() in BCIModel::process(const std::any&) to avoid a window where a cancel() is dropped on the floor. - _addonOutputCallback now unpacks transcript arrays so response.await() yields flat segments (matches TranscriptionWhispercpp). - examples/transcribe-neural.js identical-branch ternary fixed. - README broken whisper.cpp link fixed; docs/BCI_V184_COMPAT.md stale overlay commit ref updated. - Integration test honours BCI_REQUIRE_MODEL=1 to turn missing-model into a loud failure for CI (default behaviour unchanged: local dev still skips). - index.d.ts now imports QvacResponse from @qvac/infer-base/src/QvacResponse and LoggerInterface from @qvac/logging instead of hand-rolling them. Tests - Clean rebuild from scratch (rm -rf build prebuilds && bare-make generate/build/install) succeeds. - npm run lint: clean (now covers lib/**). - npm run test:dts: clean. - npm run test:integration: 3/3 pass, 10/10 asserts, 6.0% average WER (matches baseline). - npm run test:cpp: 18/18 pass (was 7; +11 new tests covering unknown-key rejection, numeric double-to-int coercion, range validation, ContextGpuDevice bounds, passthrough mode, invalid embedder handling). - bare examples/transcribe-neural.js --batch: 5/5 samples, 6.0% avg WER. - bare examples/transcribe-neural.js test/fixtures/neural_sample_0.bin: output unchanged ("You can see the good at this point as well."). Made-with: Cursor
GustavoA1604
requested changes
Apr 21, 2026
GustavoA1604
requested changes
Apr 21, 2026
GustavoA1604
left a comment
Contributor
There was a problem hiding this comment.
Please keep the package name bci-whispercpp. We are going to rename the other qvac-lib-infer later
added 4 commits
April 21, 2026 14:09
… validation Move the addon package back to packages/bci-whispercpp, remove unneeded overlay/docs files requested in review, and tighten JS/C++ lifecycle/config safety checks to prevent invalid-state and malformed-input issues. Made-with: Cursor
… config - Replace cd && chain in test:cpp:run with direct path (CLAUDE.md compliance) - Route whisper_log_set through addon-cpp logger instead of silencing with once_flag, preventing inter-addon log handler clobber when BCI and transcription-whispercpp coexist in the same process - Fix stats heuristic in bci.js _addonOutputCallback to match actual BCIModel::runtimeStats keys (tokensPerSecond/totalWallMs, not the audio-addon keys audioDurationMs/totalSamples) - Drain _inferenceQueueWaiter in unload()/destroy() before calling destroyInstance(), closing the race where destroy could fire while process() is mid-execution on the native thread - Remove auto-load in BCIModel::process — throw immediately if context is null instead of lazy-loading outside the controlled lifecycle - Remove dead set_weights_for_file snake_case stub and unused <span> - Add qvac-lint-cpp to vcpkg.json dependencies (matches all peer addons) - Remove empty qvac-lint-cpp overlay directory (per Gustavo review) - Remove stale bci_wer/bci_transcription from manifest.json - Stop gitignoring package-lock.json (match monorepo convention) - Move computeWER into BCIWhispercpp namespace in index.d.ts - Downgrade @types/node to ^22.15.3, remove bare-fs from devDeps - Fix PR template code blocks from typescript to javascript Made-with: Cursor
…red errors, lifecycle safety Align bci-whispercpp with monorepo conventions and fix code quality issues found during thorough review of the POC implementation. Build/config: - .gitignore aligned with peer addons (package-lock.json, .npmrc, IDE files, vcpkg cache, generated test bundles) - vcpkg.json: use "version" instead of deprecated "version-string" - package.json: replace $(find) in lint-cpp with explicit file list, remove unused bare-stream/bare-tty deps, add bare-fs to production deps - CHANGELOG.md: add date per Keep a Changelog format JS fixes: - Move fs.existsSync model check from constructor to _load(), matching TranscriptionWhispercpp lifecycle pattern - Remove dead PAUSED/STOPPED state enum values from bci.js - Add explicit event name matching alongside heuristic fallback in _addonOutputCallback (matches peer whisper.js pattern with BCI stat keys) - Add miscConfig.caption_enabled boolean type validation in configChecker - Extract duplicated flattenSegments into shared lib/util.js - Fix index.d.ts import from fragile internal path to stable @qvac/infer-base C++ fixes: - Guard whisper_log_set with std::once_flag to prevent clobbering log handlers from coexisting whisper-based addons in the same process - Replace std::runtime_error with structured StatusError/bci_error::makeStatus in BCIModel::load() and loadEmbedderIfNeeded() for proper JS error mapping - Use std::move in process(const std::any&) to avoid copying multi-MB neural signal buffers on every inference call Made-with: Cursor
- Add qvac-lint-cpp configure_file block to CMakeLists.txt (copies .clang-format, .clang-tidy, .valgrind.supp from vcpkg into build tree, matching qvac-lib-infer-whispercpp pattern) - Extend lint-cpp script to cover all .hpp header files - Match peer index.d.ts QvacResponse import path (deep import from @qvac/infer-base/src/QvacResponse) - Replace brittle string-matching in _isConfigurationError with structured error detection (TypeError, ERR_ASSERTION code checks) - Remove stale configChecker comments about unimplemented BCI params (smooth_kernel_std, smooth_kernel_size, sample_rate) - Remove unused error codes: FAILED_TO_GET_STATUS, FAILED_TO_RESET, FAILED_TO_PAUSE and their addCodes registrations - Remove unused K_SAMPLES_PER_SECOND constant from BCIModel.cpp - Remove unused <span> include from AddonJs.hpp - Add qvac-lib-inference-addon-cpp to NOTICE C++ dependencies - Add cpp-test-results.xml to .gitignore Made-with: Cursor
added 2 commits
April 21, 2026 17:18
The BCI patches (variable conv1 kernel, windowed attention) are now merged into tetherto/qvac-ext-lib-whisper.cpp master and tagged as v1.8.4.2. The local overlay that pinned a specific fork commit is no longer needed. - Delete vcpkg-overlays/whisper-cpp/ (portfile.cmake + vcpkg.json) - Remove VCPKG_OVERLAY_PORTS from CMakeLists.txt - Bump whisper-cpp override from 1.8.4 to 1.8.4.2 - Point vcpkg-configuration.json at personal fork registry (sharmaraju352/qvac-registry-vcpkg) temporarily until tetherto/qvac-registry-vcpkg#125 merges, then swap back - Update README whisper.cpp patches section Verified: clean build from scratch + 18/18 C++ tests + 3/3 integration tests (10/10 asserts, 6.0% avg WER) + batch example all pass. Made-with: Cursor
Registry PR tetherto/qvac-registry-vcpkg#125 has been merged. Swap vcpkg-configuration.json from the personal fork back to the upstream tetherto/qvac-registry-vcpkg and update the baseline to the merge commit. Verified: clean build from scratch + all tests pass on both bci-whispercpp (18/18 C++, 3/3 integration, 6.0% WER) and transcription-whispercpp (106/106 C++, 28/28 unit, 10/10 integration, all extended suites). Made-with: Cursor
ogad-tether
previously approved these changes
Apr 21, 2026
ishanvohra2
previously approved these changes
Apr 21, 2026
GustavoA1604
requested changes
Apr 21, 2026
- Reset is_warmed_up_ in BCIModel::unload() so re-load triggers warmup - Add FailedToLoadModel and EmbedderWeightsNotFound error codes to BCIErrors.hpp; use them instead of InvalidNeuralSignal for context init failure (BCIModel.cpp:116) and missing embedder (BCIModel.cpp:90) - Wrap addon.activate() in try-catch in index.js _load(), throwing FAILED_TO_ACTIVATE with structured error on failure - Make all JS error codes sequential (26001-26013, no gaps) Made-with: Cursor
8513287
GustavoA1604
requested changes
Apr 22, 2026
GustavoA1604
approved these changes
Apr 22, 2026
ishanvohra2
approved these changes
Apr 22, 2026
Proletter
pushed a commit
that referenced
this pull request
May 24, 2026
…s review Align the BCI package with the inference-addon family conventions and resolve the review findings that accumulated across PR #1583. Breaking changes - Package directory renamed from packages/bci-whispercpp to packages/qvac-lib-infer-bci-whispercpp (npm name @qvac/bci-whispercpp unchanged). - Error codes moved from 7001-7013 (collided with @qvac/tts-onnx and the @qvac/transcription-parakeet fallback range) to the dedicated 26001-27000 range. Also adds FAILED_TO_START_JOB, INVALID_CONFIG, and EMBEDDER_WEIGHTS_INVALID for cases that were previously swallowed. Pattern / standard alignment with peer addons - Add addonLogging.js + addonLogging.d.ts + ./addonLogging subpath export. - Add CHANGELOG.md, PULL_REQUEST_TEMPLATE.md, tsconfig.dts.json. - Pin qvac-lib-inference-addon-cpp vcpkg dep to 1.1.5#1 (port-version). - vcpkg default-registry switched from git@github.com: to https:// (fixes anonymous clones and CI runners without an SSH deploy key). - Lint glob now covers lib/**/*.js. - bare engine bumped from >=1.19.0 to >=1.24.0 to match llamacpp-llm/embed. - VCPKG_OVERLAY_TRIPLETS set unconditionally and preserves external value. - Remove test:unit script that pointed at a non-existent dir; add build:pack, lint-cpp, test:dts scripts matching peer conventions. - package.json files array now includes README.md, CHANGELOG.md, and addonLogging artifacts; repository.directory + homepage point at the renamed path. PR review fixes (Gustavo, ogad-tether, github-code-quality bot) - day_idx default aligned: C++ runtime default is now 0 (matches the public JS/TS docs and NeuralProcessor header default). - BCIInterface.runJob rewrap now uses FAILED_TO_START_JOB instead of the misleading FAILED_TO_APPEND; input is validated (Uint8Array, non-empty). - day_idx: -1 passthrough mode is now explicitly documented in configChecker, README, and index.d.ts, and values < -1 are rejected at the JS boundary. - JS _load no longer sets suppress_nst/temperature defaults that fought the BCI-tuned C++ defaults in toWhisperFullParams. - Duplicate checkConfig call in BCIWhispercpp._load removed; validation now happens once inside the BCIInterface constructor. - whisper_log_set guarded by std::once_flag so it does not clobber any log handler a coexisting whisper-based addon installed in the same process. - Embedder weight loader now checks the stream state after every read and returns false on truncation instead of silently marking the weights as loaded and producing garbage at inference time. - NeuralProcessor day projection is now memoized per day_idx; same-day batch inference no longer rebuilds the O(nf^2 * r) dense matrix. - cancelRequested_.store(false) now runs before reset() in BCIModel::process(const std::any&) to avoid a window where a cancel() is dropped on the floor. - _addonOutputCallback now unpacks transcript arrays so response.await() yields flat segments (matches TranscriptionWhispercpp). - examples/transcribe-neural.js identical-branch ternary fixed. - README broken whisper.cpp link fixed; docs/BCI_V184_COMPAT.md stale overlay commit ref updated. - Integration test honours BCI_REQUIRE_MODEL=1 to turn missing-model into a loud failure for CI (default behaviour unchanged: local dev still skips). - index.d.ts now imports QvacResponse from @qvac/infer-base/src/QvacResponse and LoggerInterface from @qvac/logging instead of hand-rolling them. Tests - Clean rebuild from scratch (rm -rf build prebuilds && bare-make generate/build/install) succeeds. - npm run lint: clean (now covers lib/**). - npm run test:dts: clean. - npm run test:integration: 3/3 pass, 10/10 asserts, 6.0% average WER (matches baseline). - npm run test:cpp: 18/18 pass (was 7; +11 new tests covering unknown-key rejection, numeric double-to-int coercion, range validation, ContextGpuDevice bounds, passthrough mode, invalid embedder handling). - bare examples/transcribe-neural.js --batch: 5/5 samples, 6.0% avg WER. - bare examples/transcribe-neural.js test/fixtures/neural_sample_0.bin: output unchanged ("You can see the good at this point as well."). Made-with: Cursor
Proletter
pushed a commit
that referenced
this pull request
May 24, 2026
#1583) * QVAC-17057 feat: add bci-whispercpp package for BCI neural signal transcription Add a new @qvac/bci-whispercpp addon that transcribes brain-computer interface neural signals into text using a modified whisper.cpp backend. This POC includes: - C++ native addon with BCI model inference (NeuralProcessor, BCIModel, BCIConfig) built on the qvac addon-cpp framework - CMake + vcpkg build system with whisper-cpp overlay ports carrying BCI-specific patches (variable conv1 kernel, windowed attention) - JavaScript API: BCIWhispercpp class with batch transcribeFile/transcribe - Integration tests for load/destroy and batch transcription - Example script and model conversion tooling - WER utility for accuracy measurement Streaming transcription will be added in a follow-up PR (QVAC-17062). Made-with: Cursor * fix[api](bci): address review feedback, refactor to infer-base pattern, fix Linux linkage - Refactor BCIWhispercpp to use createJobHandler + exclusiveRunQueue from @qvac/infer-base instead of manual promise plumbing, matching the TranscriptionWhispercpp / LlmLlamacpp addon pattern - Constructor now takes { files: { model }, logger, opts } (was { modelPath }) - transcribe/transcribeFile return QvacResponse - Add unload(), getState(), exclusiveRunQueue-serialized destroy() - Add @qvac/infer-base dependency Address all review feedback from Gustavo (PR #1583): - Remove unused END_OF_INPUT, totalSamples_, sleep_for(1ms) - Use QvacErrorAddonBCI for model-not-found, add BUFFER_LIMIT_EXCEEDED - Fix n_threads/duration_ms double→int conversion in BCIConfig.cpp - Add bounds validation for all BCIConfig numeric params - Throw on unknown config keys (was silently ignored) - Consume gpu_device in context params - Collect whisper timings in runtimeStats() - Trim unused BCIErrors enum values, map codes to distinct names - Add MAX_BUFFERED_BYTES guard and nextSafeId in bci.js - Fix _activeJobId race: set after native acceptance - Remove unimplemented bciConfig params from JS whitelist + index.d.ts - Promote hardcoded kernel-trim threshold to named constant - Pre-allocate dummyAudioPad_ as class member (avoid repeated allocs) - Rename bci-addon.test.js → addon.test.js - Replace t.skip() with proper assertions - Fix day_idx handling in tests/examples (group by day, pass to config) - Generate comprehensive NOTICE file - Update vcpkg overlay to v1.8.4 description Fix Linux C++ test linkage: - Add vcpkg triplets (x64-linux, arm64-linux) with -stdlib=libc++ - Add linux-clang toolchain (clang-19) - Set VCPKG_OVERLAY_TRIPLETS in CMakeLists.txt for Linux builds Made-with: Cursor * perf(bci): bump whisper-cpp overlay to include mask caching and per-layer flash attn Update whisper-cpp overlay to 5645ad60 which includes: - Cached window_mask recompute for exp_n_audio_ctx overrides - Per-layer flash attention (upper encoder layers use FA even with BCI) - std::abs instead of C abs in mask computation Made-with: Cursor * chore(bci): bump whisper-cpp overlay to include jpgaribotti review fixes Update overlay to tetherto/qvac-ext-lib-whisper.cpp@3e91e3a4 which addresses jpgaribotti's review on PR #10: 1. Extract compute_window_mask() helper to eliminate duplicated O(n_ctx^2) mask fill logic 2. Guard encode-time mask block with hparams.is_bci 3. Add is_bci to graph builder window_mask guard 4. Validate BCI hparams (conv1_kernel > 0, window_size >= 0) 5. Document n_mels > 256 threshold convention Bump port-version to 3. Made-with: Cursor * fix(bci): add test fixture download to download-models.sh Address Gustavo's review feedback: test fixtures (neural_sample_*.bin) are gitignored but the PR had no way for developers to obtain them. Rewrite download-models.sh to fetch both models and test fixtures from the bci-test-assets-v0.1.0 GitHub release. Supports --models, --fixtures, or both (default). Made-with: Cursor * fix(bci): address review findings — version mismatch, test indexing, cleanup - Bump whisper-cpp override in vcpkg.json from 1.7.5.1 to 1.8.4 to match the overlay port version - Move gtest to a vcpkg "tests" feature so it is only pulled when BUILD_TESTING=ON - Fix PaddedFramesAreZero test: use mel-major indexing (data[bin * n_frames + frame]) matching the actual processToMel layout - Remove four unused overlay patch files (0001–0004) now that portfile.cmake fetches from the tetherto fork with patches baked in - Add TODO comment in download-models.sh noting the temporary personal fork for release assets Made-with: Cursor * fix(bci): address review findings — race guard, cross-platform path, docs accuracy - Wrap transcribe() in exclusiveRunQueue to prevent race between inference and unload/destroy - Use find_last_of("/\\") in loadEmbedderIfNeeded for Windows compat - Add empty-buffer guard in bci.js append() before end-of-job - Update download-models.sh to use tetherto/qvac release repo - Add transformers to NOTICE and README model conversion prerequisites - Fix README WER table to match actual live test results (6.0% avg) - Fix BCI_V184_COMPAT.md stale test filename and overlay ref - Remove unused bci_wer_vs_expected field from manifest.json - Update whisper.cpp patches section to reflect fork-based overlay Made-with: Cursor * fix(bci): harden lifecycle, type safety, and C++ code quality - Fix unload/destroy race: call destroyInstance() before _job.fail() so the native side stops before the JS job is failed, and remove redundant cancel() call (destroyInstance already cancels internally) - Wrap BCIInterface construction in try/catch so a native init failure sets addon=null and throws a structured QvacErrorAddonBCI - Change JSAdapter loadContextParams/loadMiscParams/loadBCIParams to return void (callers already mutate via reference, return was dead) - Add dayIdx bounds-check warning in BCIModel::process when the value falls outside [0, numDays-1] before silent clamping - Promote hardcoded gaussian smoothing params (std=2.0, kernel=100) to named constants K_SMOOTH_KERNEL_STD / K_SMOOTH_KERNEL_SIZE - Add NeuralProcessor::getNumDays() accessor for the bounds check - Remove [key: string]: unknown escape hatch from WhisperConfig in index.d.ts; enumerate all valid keys explicitly - Fix test:cpp:run script to use direct path instead of cd && chain Made-with: Cursor * chore(bci): point whisper-cpp overlay to merged master (2b1e04f) qvac-ext-lib-whisper.cpp PR #10 has been merged. Update the overlay to reference the merge commit on master instead of the feature branch commit, so the overlay remains valid if the branch is deleted. Bump port-version to 4. Made-with: Cursor * fix(bci): serialize inference lifetime and export low-level subpaths Address ogad-tether review feedback on PR #1583: 1. Inference queue: transcribe() now holds its slot until the response settles via _enqueueInference(), matching the pattern from TranscriptionWhispercpp._enqueueExclusiveRunResponse(). Previously the exclusiveRunQueue released the slot as soon as runJob() was accepted, allowing a second concurrent transcribe() to race in and either clobber the first response or get rejected by the native side. 2. Exports map: add ./bci, ./bci.js, and ./binding subpath exports so the low-level BCIInterface API documented in the README is accessible after publish. The exports map previously only exposed ./binding.js, blocking require('@qvac/bci-whispercpp/bci'). Made-with: Cursor * refactor[bc](bci): rename to qvac-lib-infer-bci-whispercpp and address review Align the BCI package with the inference-addon family conventions and resolve the review findings that accumulated across PR #1583. Breaking changes - Package directory renamed from packages/bci-whispercpp to packages/qvac-lib-infer-bci-whispercpp (npm name @qvac/bci-whispercpp unchanged). - Error codes moved from 7001-7013 (collided with @qvac/tts-onnx and the @qvac/transcription-parakeet fallback range) to the dedicated 26001-27000 range. Also adds FAILED_TO_START_JOB, INVALID_CONFIG, and EMBEDDER_WEIGHTS_INVALID for cases that were previously swallowed. Pattern / standard alignment with peer addons - Add addonLogging.js + addonLogging.d.ts + ./addonLogging subpath export. - Add CHANGELOG.md, PULL_REQUEST_TEMPLATE.md, tsconfig.dts.json. - Pin qvac-lib-inference-addon-cpp vcpkg dep to 1.1.5#1 (port-version). - vcpkg default-registry switched from git@github.com: to https:// (fixes anonymous clones and CI runners without an SSH deploy key). - Lint glob now covers lib/**/*.js. - bare engine bumped from >=1.19.0 to >=1.24.0 to match llamacpp-llm/embed. - VCPKG_OVERLAY_TRIPLETS set unconditionally and preserves external value. - Remove test:unit script that pointed at a non-existent dir; add build:pack, lint-cpp, test:dts scripts matching peer conventions. - package.json files array now includes README.md, CHANGELOG.md, and addonLogging artifacts; repository.directory + homepage point at the renamed path. PR review fixes (Gustavo, ogad-tether, github-code-quality bot) - day_idx default aligned: C++ runtime default is now 0 (matches the public JS/TS docs and NeuralProcessor header default). - BCIInterface.runJob rewrap now uses FAILED_TO_START_JOB instead of the misleading FAILED_TO_APPEND; input is validated (Uint8Array, non-empty). - day_idx: -1 passthrough mode is now explicitly documented in configChecker, README, and index.d.ts, and values < -1 are rejected at the JS boundary. - JS _load no longer sets suppress_nst/temperature defaults that fought the BCI-tuned C++ defaults in toWhisperFullParams. - Duplicate checkConfig call in BCIWhispercpp._load removed; validation now happens once inside the BCIInterface constructor. - whisper_log_set guarded by std::once_flag so it does not clobber any log handler a coexisting whisper-based addon installed in the same process. - Embedder weight loader now checks the stream state after every read and returns false on truncation instead of silently marking the weights as loaded and producing garbage at inference time. - NeuralProcessor day projection is now memoized per day_idx; same-day batch inference no longer rebuilds the O(nf^2 * r) dense matrix. - cancelRequested_.store(false) now runs before reset() in BCIModel::process(const std::any&) to avoid a window where a cancel() is dropped on the floor. - _addonOutputCallback now unpacks transcript arrays so response.await() yields flat segments (matches TranscriptionWhispercpp). - examples/transcribe-neural.js identical-branch ternary fixed. - README broken whisper.cpp link fixed; docs/BCI_V184_COMPAT.md stale overlay commit ref updated. - Integration test honours BCI_REQUIRE_MODEL=1 to turn missing-model into a loud failure for CI (default behaviour unchanged: local dev still skips). - index.d.ts now imports QvacResponse from @qvac/infer-base/src/QvacResponse and LoggerInterface from @qvac/logging instead of hand-rolling them. Tests - Clean rebuild from scratch (rm -rf build prebuilds && bare-make generate/build/install) succeeds. - npm run lint: clean (now covers lib/**). - npm run test:dts: clean. - npm run test:integration: 3/3 pass, 10/10 asserts, 6.0% average WER (matches baseline). - npm run test:cpp: 18/18 pass (was 7; +11 new tests covering unknown-key rejection, numeric double-to-int coercion, range validation, ContextGpuDevice bounds, passthrough mode, invalid embedder handling). - bare examples/transcribe-neural.js --batch: 5/5 samples, 6.0% avg WER. - bare examples/transcribe-neural.js test/fixtures/neural_sample_0.bin: output unchanged ("You can see the good at this point as well."). Made-with: Cursor * fix[api](bci): restore bci-whispercpp package path and harden runtime validation Move the addon package back to packages/bci-whispercpp, remove unneeded overlay/docs files requested in review, and tighten JS/C++ lifecycle/config safety checks to prevent invalid-state and malformed-input issues. Made-with: Cursor * fix[api](bci): address code review findings across JS, C++, and build config - Replace cd && chain in test:cpp:run with direct path (CLAUDE.md compliance) - Route whisper_log_set through addon-cpp logger instead of silencing with once_flag, preventing inter-addon log handler clobber when BCI and transcription-whispercpp coexist in the same process - Fix stats heuristic in bci.js _addonOutputCallback to match actual BCIModel::runtimeStats keys (tokensPerSecond/totalWallMs, not the audio-addon keys audioDurationMs/totalSamples) - Drain _inferenceQueueWaiter in unload()/destroy() before calling destroyInstance(), closing the race where destroy could fire while process() is mid-execution on the native thread - Remove auto-load in BCIModel::process — throw immediately if context is null instead of lazy-loading outside the controlled lifecycle - Remove dead set_weights_for_file snake_case stub and unused <span> - Add qvac-lint-cpp to vcpkg.json dependencies (matches all peer addons) - Remove empty qvac-lint-cpp overlay directory (per Gustavo review) - Remove stale bci_wer/bci_transcription from manifest.json - Stop gitignoring package-lock.json (match monorepo convention) - Move computeWER into BCIWhispercpp namespace in index.d.ts - Downgrade @types/node to ^22.15.3, remove bare-fs from devDeps - Fix PR template code blocks from typescript to javascript Made-with: Cursor * fix[api](bci): address review findings — standards alignment, structured errors, lifecycle safety Align bci-whispercpp with monorepo conventions and fix code quality issues found during thorough review of the POC implementation. Build/config: - .gitignore aligned with peer addons (package-lock.json, .npmrc, IDE files, vcpkg cache, generated test bundles) - vcpkg.json: use "version" instead of deprecated "version-string" - package.json: replace $(find) in lint-cpp with explicit file list, remove unused bare-stream/bare-tty deps, add bare-fs to production deps - CHANGELOG.md: add date per Keep a Changelog format JS fixes: - Move fs.existsSync model check from constructor to _load(), matching TranscriptionWhispercpp lifecycle pattern - Remove dead PAUSED/STOPPED state enum values from bci.js - Add explicit event name matching alongside heuristic fallback in _addonOutputCallback (matches peer whisper.js pattern with BCI stat keys) - Add miscConfig.caption_enabled boolean type validation in configChecker - Extract duplicated flattenSegments into shared lib/util.js - Fix index.d.ts import from fragile internal path to stable @qvac/infer-base C++ fixes: - Guard whisper_log_set with std::once_flag to prevent clobbering log handlers from coexisting whisper-based addons in the same process - Replace std::runtime_error with structured StatusError/bci_error::makeStatus in BCIModel::load() and loadEmbedderIfNeeded() for proper JS error mapping - Use std::move in process(const std::any&) to avoid copying multi-MB neural signal buffers on every inference call Made-with: Cursor * fix[api](bci): align with peer addon standards and remove unused code - Add qvac-lint-cpp configure_file block to CMakeLists.txt (copies .clang-format, .clang-tidy, .valgrind.supp from vcpkg into build tree, matching qvac-lib-infer-whispercpp pattern) - Extend lint-cpp script to cover all .hpp header files - Match peer index.d.ts QvacResponse import path (deep import from @qvac/infer-base/src/QvacResponse) - Replace brittle string-matching in _isConfigurationError with structured error detection (TypeError, ERR_ASSERTION code checks) - Remove stale configChecker comments about unimplemented BCI params (smooth_kernel_std, smooth_kernel_size, sample_rate) - Remove unused error codes: FAILED_TO_GET_STATUS, FAILED_TO_RESET, FAILED_TO_PAUSE and their addCodes registrations - Remove unused K_SAMPLES_PER_SECOND constant from BCIModel.cpp - Remove unused <span> include from AddonJs.hpp - Add qvac-lib-inference-addon-cpp to NOTICE C++ dependencies - Add cpp-test-results.xml to .gitignore Made-with: Cursor * chore(bci): remove whisper-cpp overlay, consume v1.8.4.2 from registry The BCI patches (variable conv1 kernel, windowed attention) are now merged into tetherto/qvac-ext-lib-whisper.cpp master and tagged as v1.8.4.2. The local overlay that pinned a specific fork commit is no longer needed. - Delete vcpkg-overlays/whisper-cpp/ (portfile.cmake + vcpkg.json) - Remove VCPKG_OVERLAY_PORTS from CMakeLists.txt - Bump whisper-cpp override from 1.8.4 to 1.8.4.2 - Point vcpkg-configuration.json at personal fork registry (sharmaraju352/qvac-registry-vcpkg) temporarily until tetherto/qvac-registry-vcpkg#125 merges, then swap back - Update README whisper.cpp patches section Verified: clean build from scratch + 18/18 C++ tests + 3/3 integration tests (10/10 asserts, 6.0% avg WER) + batch example all pass. Made-with: Cursor * chore(bci): point vcpkg registry back to tetherto upstream Registry PR tetherto/qvac-registry-vcpkg#125 has been merged. Swap vcpkg-configuration.json from the personal fork back to the upstream tetherto/qvac-registry-vcpkg and update the baseline to the merge commit. Verified: clean build from scratch + all tests pass on both bci-whispercpp (18/18 C++, 3/3 integration, 6.0% WER) and transcription-whispercpp (106/106 C++, 28/28 unit, 10/10 integration, all extended suites). Made-with: Cursor * fix(bci): address Gustavo review — error types, lifecycle, error codes - Reset is_warmed_up_ in BCIModel::unload() so re-load triggers warmup - Add FailedToLoadModel and EmbedderWeightsNotFound error codes to BCIErrors.hpp; use them instead of InvalidNeuralSignal for context init failure (BCIModel.cpp:116) and missing embedder (BCIModel.cpp:90) - Wrap addon.activate() in try-catch in index.js _load(), throwing FAILED_TO_ACTIVATE with structured error on failure - Make all JS error codes sequential (26001-26013, no gaps) Made-with: Cursor * Remove date from changelog --------- Co-authored-by: Raju <raju.sharma> Co-authored-by: Ishan Vohra <ishanvohra2@gmail.com> Co-authored-by: GustavoA1604 <54457676+GustavoA1604@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
@qvac/bci-whispercpppackage for brain-computer interface neural signal transcription using a modified whisper.cpp backendtetherto/qvac-ext-lib-whisper.cppv1.8.4)BCIWhispercppclass (usingcreateJobHandler+exclusiveRunQueuefrom@qvac/infer-base) with batchtranscribeFile/transcribereturningQvacResponseResults
Test plan
npm run buildsucceeds (native addon compiles with whisper-cpp v1.8.4 from tetherto fork)bare examples/transcribe-neural.js test/fixtures/neural_sample_0.binproduces transcription outputbare examples/transcribe-neural.js --batch— all 5 samples transcribed, 6.0% average WERnpm run test:integrationpasses (3/3 tests, 10/10 assertions — load/destroy + batch transcription + WER measurement)npm run test:cpppasses (7/7 GoogleTest tests including mel layout validation)npm run lintpasses (standardjs)Made with Cursor